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PILOTS’ USE OF A TRAFFIC ALERT AND COLLISION- AVOIDANCE 
SYSTEM (TCAS II) IN SIMULATED AIR CARRIER OPERATIONS 

Volume I: Methodology, Summary, and Conclusions 


EXECUTIVE SUMMARY 


In response to requests from the Federal Aviation Administration (FAA) and the air carrier industry, 
NASA Ames Research Center and FAA investigators designed and carried out an experiment to evalu- 
ate the use by air carrier pilots of traffic alert and collision-avoidance system (TCAS II) equipment 
(referred to hereafter as TCAS) during simulated line operations. 

Approach 

This study utilized full-mission simulations of eight air carrier flights. Sixteen three-person airline 
flight crews currently flying Boeing 727 aircraft served as subjects. Each crew flew eight flight seg- 
ments during a 10-hr simulated duty day. They were exposed to potential and actual conflicts with 
other aircraft under daylight conditions of varying visibility, and twilight or night ambient illumination. 

Each crew flew the simulation under one of four conditions: (1) a control condition without TCAS 
equipment; (2) a minimal TCAS configuration without a display of conflicting traffic; (3) a TCAS with 
a display on which traffic was presented only when a conflict occurred; and (4) a TCAS with a full- 
time display of traffic in the vicinity of the 727. The latter two conditions are approximately 
equivalent to display configurations being utilized in the TCAS limited installation program scheduled 
to begin on three U.S. air carriers early in 1988. 

The crews were given aircraft differences training, simulator familiarization, and instruction in the use 
of the TCAS equipment to which they were assigned on the day before their experimental flights. The 
experiment day covered approximately 10 hr of duty; eight segments were flown on a schedule which 
involved approximately 6 hr of flight (scheduled block time 6:48). All crews flew identical segments 
and all were exposed to the same conflicting aircraft under similar conditions. All flights were con- 
ducted in a simulated, full-air-traffic-control environment 


Digital, audio, and video data were recorded, reduced, and subjected to graphical and statistical ana- 
lyses. The results are briefly summarized here; the body of the report contains a detailed description of 
the findings. 


Findings 


This study was one of several (see references) designed to assess the impact of TCAS. An implicit 
question in all of these studies has been whether TCAS could be demonstrated to have a beneficial 
effect on the safety of aircraft operations, as measured by the ability of pilots of TCAS-equipped air- 
craft to avoid serious traffic conflicts in simulated line operations. 

In this experiment, there was a significant difference between non-TCAS and TCAS crews with respect 
to outcomes of the conflicts to which all were exposed. Without TCAS, in four instances minimum 
separation between aircraft was less than 1000 ft horizontal and 200 ft vertical simultaneously during 
32 flight segments; in one case, minimum separation was less than 500 ft horizontal and 100 ft verti- 
cal. With TCAS, there were no instances in 96 flight segments in which minimum aircraft separation 
was less than 1000 ft horizontally and 200 ft vertically. Maneuvers initiated in response to TCAS 
commands increased vertical separation between conflicting aircraft in 37 of 40 cases in which a 
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detailed outcome analysis was performed. (1 and 4) 1 (Also see appendix Q) 

Flight crew response times were well within the 5 sec allocated by the TCAS logic, in only one of 57 
advisories requiring a maneuver was this limit exceeded (by 2 sec). The average response time was 
1.9 sec. All maneuvers were in the commanded direction. (2 and 3) 

Perceived workload was evaluated by eac h crew member after each flight. As has been the case in 
most previous studies, crewmembers rated their workload as significantly higher when they were con- 
trolling the aircraft. Differences in workload between control (non-TCAS) and TCAS conditions were 
not significant; but the differences for first officers approached significance, condition 3 being rated by 
them as involving the lowest and condition 4 the highest workload. (19) 

Pilots with TCAS tended to exceed the altitude changes required by the TCAS logic; the average 
required change was 512 ft, the average observed change was 652 ft. In 9 of 19 cases, pilots changed 
altitude unnecessarily, by an average of 206 ft, when only a preventive (no altitude change) advisory 
was presented by the equipment. (3, 4, and 8) 

There were no differences in response times to maneuver advisories as a function of the amount of 
traffic information available; but crews without the planform display of traffic (condition 2) tended to 
maneuver slightly more abruptly, with greater peak rates of climb or descent, when a resolution 
advisory was presented. (5 and 6) 

Pilots initiated avoidance maneuvers more promptly when in a climb or descent than when in level 
flight. They attained the commanded rates of climb or descent most promptly when reducing their rate 
of descent, slightly less promptly when entering a climb, even less promptly when entering a descent, 
and least promptly when reducing their rate of climb. (7 and 17) 

The probability of visual acquisition of conflicting aircraft did not vary as a function either of the pres- 
ence of TCAS or of the information level within TCAS conditions. (9) 

The TCAS operating procedures prescribed in this experiment stated that pilots were permitted to 
maneuver without regard to TCAS commands when the conflicting aircraft was idendfied visually. 
Subject pilots were instructed, however, that such maneuvers may invalidate the ability of TCAS to 
assist in conflict resolution. No differences were observed in pilot behavior as a function of visual 
acquisition of conflicting traffic, except that pilots under condition 3 took longer to establish com- 
manded rates of climb or descent when the traffic was in view (as they had the right to do under these 
rules). (10 and 11) Pilots were as prompt to respond under instrument as under visual meteorological 
conditions, and there were no measured differences in their responses. (12 and 13) 

The TCAS logic may command a maneuver toward another aircraft’s present altitude when one or both 
aircraft are climbing or descending; this was announced in the auditory message that accompanies a 
resolution advisory, and crews were given specific instruction with regard to these crossing maneuvers. 
There has been concern that such maneuvers might be worrisome to pilots; however, crossing 
maneuvers were specifically examined in this study. It was found that pilots responded significantly 
more slowly to crossing maneuvers, though average response times were still well within the 5 sec 
limit Peak vertical velocities were also higher in crossing maneuvers, and the amount of altitude 
change during such maneuvers was significantly greater. (14) Visual acquisition of conflicting traffic 
did not affect these responses. (15) The amount of information available, however, did affect response 
times in crossing maneuvers; pilots without a planform display responded most quickly (condition 2), 


‘The numbers in parentheses refer to specific numbered research questions in the Results section. Questions 18, 20, 25, 29, and 30 are 
not addressed in this summary. 
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and those with information only during conflicts (condition 3) least rapidly, though within the TCAS 
limits. (16) 

Pilots understood that they were not to maneuver on the basis of information on the traffic displays 
unless they had visually acquired the target-that the information was designed to assist them to acquire 
conflicting traffic visually, and that anticipatory maneuvers might invalidate the TCAS logic. Nonethe- 
less, there were 14 instances in which pilots with traffic displays (conditions 3 and 4) initiated an 
avoidance maneuver based on the information shown on their displays, 7 under each condition. Seven 
of the 8 crews exposed to these two conditions made such maneuvers. Three of five turn maneuvers 
were initiated to avoid unseen mode A targets. (21) 

Pilots exposed to the non-TCAS condition took evasive action in 17 of 24 cases in which conflicting 
traffic was sighted; in nine cases, a pitch change was used, and in eight, a turn was initiated. Their 
maneuvers tended to be less abrupt than those of TCAS-equipped crews, as would be expected since 
the non-TCAS crews maneuvered only enough to insure visual separation. They were not able to see, 
and thus to maneuver to avoid, nine other of the 33 conflicting aircraft that would have triggered a 
resolution advisory, however. (22) 

Pronounced idiosyncratic differences were observed in the performance of evasive maneuvers with 
respect to response time, time to attain commanded rates of altitude change, vertical velocity overshoot, 
and time to complete a return to previous altitude or state. (23) There were no systematic performance 
differences between captains and first officers, however, nor did responses to debriefing questionnaires 
differ as a function of crew position. (24) 

Pilots were asked during debriefings to estimate their average response times to TCAS maneuver 
advisories. Their estimates were nearly twice as long as their average measured response times. (28) 
They were also asked to evaluate the simulation, and those exposed to conditions 2, 3 and 4 were 
asked to evaluate the effectiveness of the TCAS equipment. There were few statistically significant 
differences across conditions. (26 and 27) 


Summary 

Under the conditions of this simulation experiment, TCAS II was entirely effective in ameliorating the 
seriousness of traffic conflicts. The amount of information on the location of other air traffic had little 
effect on the responses of the flight crews to TCAS resolution advisories; measured responses were 
equally effective in crew members having no information, limited information, or full information 
regarding traffic in their immediate surround. 

Some pilots used the information provided on the planform displays of conflicting traffic to maneuver 
in advance of a maneuver advisory, despite instructions not to do so. While ultimate responsibility for 
safety of flight rests with the pilot in command, this matter must be emphasized in TCAS training; but 
it must be borne in mind that pilots will use all of the information available to them to ensure safety of 
flight. 

Three turning maneuvers were made to avoid unseen mode A (non-altitude reporting) targets. Crew 
comments during these maneuvers indicated that they were aware that they had been instructed not to 
make such maneuvers, but that they were also keenly aware that they would not receive maneuver 
advisories on mode A traffic. The uncertainty caused by incomplete information about such targets 
suggests the need for mode C transponding equipment in all aircraft likely to interact with TCAS- 
equipped aircraft, especially in terminal areas where aircrew and air traffic controller workload is 
already high. 
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The ability of these flight crews to detect conflicting aircraft visually appeared to be independent of the 
presence of TCAS equipment and of the level of TCAS information available. It must be recognized, 
however, that the visual system used provides at best a limited simulation of the real world in its abil- 
ity to reproduce the appearance of other traffic. 

Conclusions 

It is concluded, within the limitations of this experiment, that TCAS II can appreciably lessen the 
danger posed by conflicting air traffic, without imposing unacceptable increases in flight crew work- 
load TTie importance of mode C transponders, without which TCAS cannot resolve conflicts, must be 
emphasized. 

Careful consideration should be given to how much traffic information needs to be provided by TCAS 
within the cockpit. The addition of a planform display of the position of other traffic did not improve 
flight crew performance of avoidance maneuvers, though it unquestionably provided crews with much 
more information concerning conflicting aircraft in the environment. This information was used by 
most of the crews that had to perform avoidance maneuvers on traffic not visible to them. 
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PILOTS’ USE OF A TRAFFIC ALERT AND COLLISION-AVOIDANCE 
SYSTEM (TCAS II) IN SIMULATED AIR CARRIER OPERATIONS 

Volume I: Methodology, Summary, and Conclusions 


SUMMARY 


A study of pilots’ use of and responses to a traffic alert and collision- avoidance system (TCAS II) in 
simulated air carrier line operations is described. 

A different level of information about the location of other air traffic was presented to each of three 
groups of airline pilots during their execution of eight simulated air carrier flights. Traffic conflicts 
were generated at intervals during the flights; where appropriate, these conflicts were visible to the 
flight crews. Two of these levels represent the approaches taken by several airlines that have installed 
the collision-avoidance system for an in-service evaluation. In a fourth condition, pilots flying without 
TCAS II equipment were exposed to the same traffic conflicts. 

To ensure safe separation of aircraft, TCAS II commands a climb, or a descent, or a reduction in the 
rate of climb or descent Aircraft separation was effective when the system was in use; no aircraft 
came within 200 ft vertically and 1000 ft horizontally. 


Average response times did not differ as a result of the amount of traffic information available. 
Response accuracy, as measured by the root mean square overshoot in the rate of climb or descent, 
also showed no differences associated with the level of traffic information. Average peak overshoots 
in response varied significantly among conditions. 1) The mean for those crews with no traffic infor- 
mation was 2272 ft/min greater than the commanded rate of climb or descent. 2) Those crews 
presented with traffic information only during a conflict had a mean of 1221 ft/min. 3) Those with 
continuous traffic information averaged 1317 ft/min. These momentary peak overshoot differences, 
however, did not result in significant differences in the amount of altitude change. 

No learning effects were observed. Differences in flight experience did not appear to contribute to the 
small observed performance differences. 

Pilots who had displays of conflicting traffic used the displays to maneuver to avoid unseen traffic 
before a resolution advisory was issued by the TCAS II equipment. 

While the results of this experiment represent pilot response (on initial exposure only) to this traffic 
alert and collision-avoidance system under simulated conditions, they indicate (1) that pilots are able to 
utilize TCAS II effectively within the response times allocated by the TCAS II logic, and that (2) 
TCAS II, properly used, is effective in ameliorating the severity of the simulated traffic conflicts 
presented in this study. 

Volume I presents the methodology, research questions and results, and a summary and conclusions of 
the study. 

Volume II of the report contains appendices referenced in Volume I. The appendices provide details 
of the experiment and the results, and contain the text of two reports written in support of the program. 



INTRODUCTION 


The traffic alert and collision-avoidance system (TCAS II, referred to hereafter as TCAS) requires 
prompt and accurate pilot responses to effect avoidance maneuvers, in order to prevent midair colli- 
sions. This study examined human engineering issues related to pilots’ responses to the system. 

A major objective of the study was to determine how the performance of a flight maneuver was 
affected by the amount of precursory information provided about surrounding air traffic. That is, is a 
pilot’s response time or accuracy affected as the amount of information on other air traffic is varied? 

The collision-avoidance system represents additional information in the cockpit. The system is 
representative of many new avionics capabilities, and therefore it afforded an excellent opportunity to 
address the pervasive and expanding information transfer issues posed by advanced technology. 

Several airlines have installed collision-avoidance systems for in-service evaluations. The carriers 
involved have different philosophies regarding the amount of information needed to describe the loca- 
tion of other aircraft. They have also taken different approaches toward the circumstances under which 
this information is presented. Prior to these in-service evaluations, NASA Ames Research Center 
teamed with the Federal Aviation Administration (FAA) to study the human factors issues associated 
with the use of the TCAS system in an operational environment. 

An industry survey was performed to establish the critical human performance concerns. This was 
accomplished through interviews with researchers, program engineers, and pilots familiar with the 
TCAS. (For a comprehensive list of TCAS human engineering issues, see Society of Automotive 
Engineers, 1987.) 

The issues addressed in this study include: (1) the level of traffic location information provided to 
pilots, (2) the use of TCAS in instrument and visual meteorological conditions, (3) actions taken when 
the TCAS system is unable to resolve the traffic conflict, (4) pilot performance of avoidance maneuvers 
that require crossing through the altitude of the intruder, (5) pilot performance of avoidance maneuvers 
from level flight vs while climbing or descending, and (6) the use of TCAS with visual acquisition of 
the conflicting traffic. 

To provide background information for the study, the NASA Aviation Safety Reporting System 
(ASRS) provided information on voluntary reports of near midair collisions received from pilots and 
controllers. The ASRS definition of a near midair collision is a miss distance estimated as less than 
500 ft. An incident may also be classified as a near midair collision in some cases in which miss dis- 
tances are not specified but the crew states there was danger of collision. The database held 28,970 
reports at the time the analysis was conducted (January 29, 1986). 

In 2001 reports from pilots concerning near midair collisions, pilots reported sighting the conflicting 
traffic in 1599 instances. They took evasive action in 1279 cases. Though these numbers may be 
affected both by overestimation of the seriousness of the events and by underreporting to a voluntary 
program, it is clear that substantial numbers of reporters felt themselves threatened by conflicting air 
traffic. It is this problem that TCAS addresses by providing an independent backup both to air traffic 
control (ATQ and to the ability of pilots to see and avoid other aircraft 
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The investigators wish to acknowledge their indebtedness to Mr. Joseph Fee and the staff of the FAA 
TCAS Program Office; Mr. William Russell, Air Transport Association of America; Mr. James 
Lumsden, Bendix/King Corporation; Mr. William Lynch, Sperry Dalmo- Victor Corporation; Mr. David 
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Capt. Robert Buley, Northwest Airlines, for their support throughout the TCAS study. We are espe- 
cially indebted to the staff of the NASA Ames Man- Vehicle Systems Research Facility, whose innova- 
tiveness and hard work made it possible to implement the simulations under severe time constraints. 
Finally, the cooperation of Air Transport Association member airlines and the excellent work of the 
flight crews who participated in the experiment should be a source of pride to the air carrier industry 
and its pilot representative organizations. 


METHODOLOGY 


Subjects 

Sixteen three-person flight crews currently flying line operations in the Boeing 727 served as subjects 
for this study. The Air Transport Association provided flight crews from 1 1 member air carriers. The 
airlines were asked to send current, line-qualified pilots. The airlines varied in their method of identi- 
fying participants. Several asked for interested crew members to volunteer for the study. Two crews 
were randomly selected by NASA from fists of interested volunteers. Some crews were assigned by 
their company, as they would be to a normal trip. The subjects were all paid for their participation. 

Prior to coming to Ames Research Center, the subjects received portions of a handbook describing the 
experiment (appendix A). After arrival, each subject completed the questionnaire on flight experience 
found in appendix B. The mean age of the crew members was 41.4 yr (ranging from 24 to 55). Their 
mean flight experience was 9682 hr (1400 to 23,500) with a mean Boeing 727 flight time of 2805 hr 
(150 to 11,000) and 2862 (50 to 17,000) in their current crew position. Their mean flight time for the 
last 90 days was 157 hr (0 to 250). 

In the last 90 days, 61% of the pilots reported predominantly day flying, 16% reported predominantly 
night flying, and 23% reported neither day or night flying as predominant. The average number of 
flight segments per day in the last 90 days was 2.9, with a range of 1 to 4. The flying in the last 90 
days was reported to be predominantly long-haul by 55% of the pilots, predominantly short-haul by 
34%, and 11% of the pilots reported neither short- or long-haul flying as predominant. 

When asked whether they were a "morning person" or an "evening person," 66% stated they were 
morning people, 29% said they were evening people, and 5% had no preference. 

Experimental Design 

The experimental design for the study is described in full in appendix C. Several factors were con- 
sidered and counterbalanced in the design. They included ambient twilight or night light conditions 
(the presence or absence of a horizon can ease or make more difficult a pilot s perception of the rela- 
tive vertical motion of traffic), the pilot flying (to provide equal opportunities for visual traffic to be 
sighted by either the flying or non-flying pilot), and scenario order (flight segments varied in difficulty, 
each segment was presented equally often during the first and last half of the experiment day to bal- 
ance learning or possible fatigue effects). 
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Equipment 


TCAS II - Pilots’ use of the collision-avoidance system was evaluated using three levels of 
traffic information (tables 1 and 2). All three levels provided a light and tone that alerted the crew 
when an aircraft was within 40 sec of passing very close to them (a TCAS Traffic Advisory, TA). If 
the aircraft continued to pose a threat when it was 20-25 sec away, the system notified the crew either 
to maneuver or to continue their present flight path with some restrictions (a TCAS Resolution 
Advisory, RA). This advisory was visually presented on the instantaneous vertical speed indicator 
(IVSI) (fig 1). Lighted amber segments around the outer edge of the instrument indicated the vertical 
rates which must be avoided. For example, a 2000 ft/min climb was indicated by lights from -6000 to 
+2000 ft/min. 





(b) TCAS with traffic display only during conflicts (condition 3) 



(c) TCAS with continuous traffic display (condition 4) 


Figure 1. Cockpit displays utilized in the TCAS experiment 
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When a maneuver was not required but an aircraft representing a threat was within 20-25 sec of its 
closest point of approach (CPA), an auditory tone sounded and the Traffic Advisory caution light was 
illuminated (table 1). When it was necessary to maneuver to avoid a collision, TCAS presented a 
two-tone sequence twice, followed by a voice command to "climb," "descend, or adjust vertical 
speed." The warning light was also illuminated (table 1). (For a description of the collision- 
avoidance system see Radio Technical Commission for Aeronautics, 1983.) 

TABLE 1. ALERT CHARACTERISTICS OF THE TCAS IN THIS EXPERIMENT 


TCAS alert 

Alert level 

Master alert 2 

Voice 3 

IVSI 

Traffic display 

maneuver 

time-critical 

red light 

eg "climb" 

arc lights 

red 

advisory 

warning 

siren 


(amber) 

target 

preventive 

caution 

amber light 


arc lights 

red 

advisory 


tone 


(amber) 

target 

traffic 

caution 

amber light 



amber 

advisory 


tone 



target 

proximate 

information 




white 

traffic 





target 

resolution 

information 


"clear of 


white 

complete 



conflict" 


target 

TCAS 

time-critical 

red light 

"unable to 

red flag 

red 

invalid 

warning 

siren 

command" 


target 


The crews were randomly assigned to experimental conditions. These conditions represented different 
levels of traffic information. Each crew was exposed to one experimental condition only. 

Condition 1: No TCAS 

Condition 2: TCAS with no traffic display 

Condition 3: TCAS with traffic display during conflicts, fixed range 

Condition 4: TCAS with continuous traffic display, pilot-selectable 

horizontal and vertical ranges 

Condition 1 was a control (no TCAS) case, designed to assess the responses of flight crews to the 
traffic conflicts without aid from the collision-avoidance system. These crews were given the same 
warnings of traffic by ATC that were provided to other crews, but they were given no additional assis- 
tance with respect to traffic for which the scenarios provided no ATC warnings. 

Condition 2 provided a minimal TCAS capability, with visual and auditory alerts, but without a display 
of traffic. Traffic and resolution advisories were generated as in conditions 3 and 4; but the crews 
flying under this condition received only the alerting function of the traffic advisory, without informa- 
tion on the location of the conflicting traffic. Table 2 describes the information available within the 
cockpit. 


^WU] continue until traffic is clear or until canceled by pressing lightybutton. Tone is on for 2 sec, off for 8 sec, siren sounds 

twice. 

3 Voice messages are continuous. Corrective resolutions requiring climb or descent through intruder’s altitude will be announced in 
the voice command, e.g., "climb to cross." 
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The crews flying under condition 3 also had a display of the location of threat aircraft and up to two 
other aircraft. This was presented only when a collision threat existed. It was a plan view, fixed 
range, cathode ray tube display with aircraft locations shown in white, amber, and red as they became 
more of a threat (fig. 1). Bearing data were subject to a ± 8° azimuth error, representative of the 
actual antenna bearing error. 

Condition 4 provided the most information about other aircraft. The location of all the aircraft being 
tracked by TCAS was depicted. Bearing data were subject to a ± 2° azimuth error. This traffic 
display had pilot-selectable horizontal and vertical ranges of 3, 5, 10, and 20 n mi and ± 2700, +7000 
-2700, and +2700 -7000 ft. This display also contained color- and shape-coded aircraft symbols. The 
information was presented continuously, even when no collision threat existed (fig. 1). 

Appendix D describes in detail the training programs the crews received, and the various levels of 
traffic information provided under conditions 2, 3 and 4. 

TABLE 2. DISPLAY CONDITIONS FOR NASA/FAA TCAS SIMULATION 


Information 

level 

Master 

alert 

Voice 

commands 

IVSI 

resolution 

Traffic 

display 

Condition l 

NA 

NA 

NA 

NA 

Condition 2 

X 

X 

X 

None 

Condition 3 

X 

X 

X 

During conflict 
only, threat + 2 
aircraft, ± 8° az- 
imuth error 

Condition 4 

X 

X 

X 

Continuously 
presented, all 

traffic ± 2700 ft, 
± 2° azimuth er- 
ror 


Note: In condition 3, the traffic was displayed for 15 sec each time the 
crew pressed the traffic switch. The condition 4 display showed traffic 
from ± 2700 ft (default) unless selected to 7000 ft above or below. 


Simulation - The study was conducted in the Man-Vehicle Systems Research Facility at NASA 
Ames Research Center. A Singer-Link Boeing 727-232 advanced technology simulator was used, with 
a six-degree-of- freedom motion simulator and a Singer-Link-Miles Image II three-channel, four- 
window, dusk-night visual system. Appendix E describes the simulation in greater detail. 
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The simulator was flown within a simulated air traffic control radar environment. Other aircraft were 
heard over the radio and seen out of the window, weather conditions permitting. These aircraft were 
under dynamic control; their initial position was preprogrammed and was triggered by the location of 
the TCAS aircraft. The air traffic included air carrier and general aviation aircraft, providing a mix of 
aircraft performance. 

The air traffic control simulation had three controller consoles and three keyboard aircraft work stations 
capable of simulating up to 36 other aircraft. All workstations and the B-727 simulator were intercon- 
nected by voice communications using appropriate air traffic control radio frequencies. The controllers 
and pilots of the keyboard aircraft used voice disguisers to simulate communications among many 
different controllers and pilots. 

The geographic area simulated in this study included Oakland and Los Angeles Air Route Traffic Con- 
trol Centers, and four terminal areas: Los Angeles, San Francisco, Sacramento, and Stockton, Califor- 
nia. Air traffic densities were appropriate for the simulated areas. The air traffic control sectors and 
radio frequencies were accurately represented in the simulation. The navigation facilities and frequen- 
cies for these areas were also accurately represented. Pilots used navigation charts provided by their 
airlines. 


Procedures 

Four flight crews were randomly assigned to each of the four experimental groups; each crew received 
only one level of traffic information. They reported for two consecutive days. The first day was train- 
ing, and the second a typical day of line flying. 

Training - After arrival, subjects filled out administrative and other questionnaires (appendices 
F & B). They received initial briefings (appendix G). The first day consisted of both classroom and 
simulator sessions for aircraft differences training. This varied from 2 to 4 hr, depending on the mag- 
nitude of the differences between the airline 727 and NASA simulator configurations. Each captain 
and first officer flew at least one instrument approach and landing. To minimize the aircraft 
differences and contribute realism, each crew used its own airline’s standard operating procedures. 
Familiarization flights were carried out in an ATC environment with other, sometimes visible, traffic. 

The differences training was followed by training on the use of the collision avoidance system. The 
crews viewed a 20-min video describing the system and demonstrating its proper use (appendix D). 
This was followed by a question and answer period, and a quiz (appendix H) to ensure understanding 
of the collision-avoidance system they would use. The quiz was reviewed until each pilot was able to 
answer each question correctly. This training program was representative of those being administered 
by the airlines for TCAS crew training. Subjects in the baseline condition (condition 1, no TCAS) 
were also shown a training tape and given a quiz. Prior to departing, crews flying conditions 2, 3 or 4 
were given the appropriate Flight Manual Supplement for TCAS (appendix I). 

Experimental Flights - On the second day, the crews repotted for a normal duty day. They 
received their flight plans, weather, and passenger loading information in the format used by their air- 
line. The airlines also provided checklists and flight manuals so that crews were able to conduct the 
flights as they would normal line operations, given the limitations of simulation. This included all 
flight duties, e.g., communications with passengers, company (radio only; ARINC communications and 
reporting system (ACARS) was not installed in the simulator), ground crews, and air traffic control. 

Scenarios - The full-mission simulation consisted of eight flights ranging in length from 30 to 
73 min. Each crew was exposed to the same scenarios including the geometry of the encounters with 
other aircraft. The scenarios were counter-balanced for order of presentation, visibility, twilight/night 
ambient illumination, workload, and captain/first officer flying. 
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Eight flight scenarios were constructed for this experiment. All the crews flew all the scenarios. All 
events presented to the subjects were scenario-dependent (i.e„ a particular target, pi air traffic control 
situation, or a malfunction was always presented in conjunction with the same specific flight leg). 

The scenarios were presented to the subjects in one of the two orders shown in table 3. The initial 
conditions for each new scenario were triggered by setting the paricing brake when the aircraft parked 
at the completion of each flight. Aside from an increase in the amount of fuel at certain stops, this 
change was not visible to the crew. 


TABLE 3. TCAS EXPERIMENT FLIGHT SCHEDULES 


Flight 

number 

Route of flight 
From To 

Schedule times 
Depart Arrive 

Flight 

time 

(min) 

Block 

time 

(min) 

Sequence 

1 


SFO 

SCK 

0830 

0902 

0:20 

0:32 

712 

SCK 

LAX 

0915 

1023 

0:58 

1:08 


LAX 

SMF 

1050 

1200 

0:59 

1:10 

713 

SMF 

SFO 

1215 

1252 

0:27 

0:37 


SFO 

SMF 

1330 

1406 

0:24 

0:36 

716 

SMF 

LAX 

1420 

1533 

1:03 

1:13 


LAX 

SCK 

1600 

1702 

0:50 

1:02 

715 

SCK 

SFO 

1715 

1745 

0:20 

0:30 

Sequence 

1— ! 

2 

« * 


LAX 

SCK 

0830 

0932 

0:50 

1:02 

715 

SCK 

SFO 

0945 

1015 

0:20 

0:30 


SFO 

SMF 

1040 

1116 

0:24 

0:36 

716 

SMF 

LAX 

1130 

1243 

1:03 

1:13 


LAX 

SMF 

1320 

1430 

0:59 

1:10 

713 

SMF 

SFO 

1445 

1522 

0:27 

0:37 


SFO 

SCK 

1550 

1622 

0:20 

0:32 

712 

SCK 

LAX 

1635 

1743 

0:58 

1:08 


The ambient light level was varied. Each scenario was presented to half the crews under twilight con- 
ditions, (visible horizon), and the other half under night levels of illumination, (no appreciable hor- 
izon). The weather conditions for each scenario were constant across subjects. 

Captains and first officers always alternated legs; the first pilot to fly was specified by the investigators 
in the pre-flight dispatch briefing. Thus half the subjects encountering each traffic conflict were cap- 
tains and half were first officers. In only one case did a crew deviate somewhat from the pre-set 
schedule. 

The scenarios varied considerably in workload induced by air traffic control, other traffic in the sur- 
round, and in one segment by a deliberate aircraft malfunction. There were short flights at lower alti- 
tudes and longer flights at high altitudes. There were two terminal areas with high traffic density and 
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two with low density. Aside from these imposed differences, crews were free to adjust their cockpit 
workload as they normally would. 

In all cases, the actual weather was similar to that forecasted. The weather varied from very good visi- 
bility with kibstantial low-level winds and turbulence at San Francisco, to fog at Stockton. The actual 
and forecast weather are shown in appendix J. 

Fuel loads were as shown in the flight planning documents; several crews asked for, and were given, 
more fuel than the minimum provided. 

The aircraft was planned to have no malfunctions, except for a single generator failure during a turn to 
final approach to Los Angeles. In a few cases, however, unplanned simulator malfunctions occurred 
and were coped with by the crew as they arose. The most serious consequence was a missed 
approach, subsequently resulting in a successful landing. 

Each scenario had a written script used by the air traffic controllers and keyboard aircraft pilots (see 
appendix K for an example). Conflict geometry is described in appendix L. The crews were able to 
hear the clearances given to the other aircraft and discern their positions and courses, as in actual 
flight. For some of the traffic, the controllers gave a traffic advisory, while other traffic was unan- 
nounced. 

Data collection - All data were immediately deidentified as to subject identity and airline 
affiliation. In addition to the computerized data, video tapes of the cockpit and the traffic display pro- 
vided information regarding the crew responses. Audio recordings of the cockpit communications, and 
the radio communications with air traffic control, provided further information. Two experimenters 
continually observed the crew via cockpit cameras and microphones. Their observations were recorded 
on the forms found in appendix M. Pilots rated the workload of each flight on the rating forms found 
in appendix N. The subjective evaluation made by the pilots was in response to the questionnaire 
found in appendix O, however further evaluation was informally conducted in a debriefing discussion 
period (appendix P). 

Performance measures - To ensure safe separation from an approaching aircraft, the collision- 
avoidance system commands a climb, a descent, or a reduction in rate of climb or descent. In this 
study, the accomplishment of these maneuvers, and the effects of the maneuvers on spacing between 
aircraft were evaluated. 

To assess pilot performance of the avoidance maneuvers prescribed by the TCAS, the following depen- 
dent measures were evaluated; (1) the type of maneuver advisory; (2) separation between aircraft at 
CPA; (3) turns or changes in vertical speed based on information from the traffic display; (4) the time 
of initial stick or throttle movement in response to the maneuver advisory; (5) the direction of initial 
response; (6) the time to attain the commanded vertical speed; (7) the root mean square of the 
difference between the actual and the commanded vertical speed; (8) the peak instantaneous vertical 
speed overshoot; (9) the time to initiate and complete a return to the previous altitude (where level) or 
to the previous vertical speed (where climbing or descending) after the maneuver, (10) the altitude 
change resulting from the maneuver, and (11) whether the intruder aircraft was visually acquired. 
Other measures were also examined, including subjective ratings of the simulation and the TCAS, ana 
pilot flight experience. 

Instrumentation - The simulated (IVSD used in this simulation were prototype instruments with 
several limitations. The eyebrow lights installed in the instruments were unable to command a 1500 
ft/min rate of climb or descent; 2000 ft/min had to be used instead of the 1500 ft/min specified for 
flight maneuvers in actual implementation of the TCAS system. For this reason, certain values 
observed in this study, (e.g., the peak vertical speed overshoot and the altitude change resulting from a 
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maneuver), are somewhat inflated due to the use of the 2000 ft/min rate of climb or descent and the 
resolution of the pilot’s instrument at that value. The time to attain the commanded vertical speed was 
calculated for 1500 ft/min for climb and descent advisories to yield a closer approximation of values 
that might be obtained in flight. 

It should also be noted that the IVSI needles, (made of lucite plastic), on these simulated instruments 
were harder to read than the needles on actual flight instruments. This may also have degraded accu- 
racy in attaining commanded speeds. 

Occurrences - The traffic conflicts presented to the flight crews were modeled on the basis of 
actual near midair collisions reported to the Aviation Safety Reporting System. They were appropriate 
for altitude, location, and phase of flight. The traffic conflicts were programmed to produce three to 
four traffic advisories per hr. With approximately 6 hr of flight this would result in 18-24 traffic 
advisories per crew. One in four of these traffic conflicts was programmed to be likely to produce a 
maneuver advisory (six per crew). 

TABLE 4. NUMBER OF TRAFFIC AND RESOLUTION ADVISORIES 
BY LEVEL OF TRAFFIC INFORMATION 


Experimental 

Traffic 

Resolution 

Ratio of 

condition 

advisories 

advisories 

TA : RA 

1. No TCAS equipment 

80 

33 

2.4 : 1 

Number per flight hr 

3.33 

1.38 


2. No traffic display 

84 

24 

3.5 : 1 

3. Conflict traffic 

65 

21 

3.1 : 1 

4. Continuous display 

85 

33 

2.6 : 1 

Total for TCAS crews 

234 

78 

3.0 : 1 

Number per flight hr 

3.25 

1.08 



Table 4 shows the numbers of traffic advisories and the number and types of resolution advisories 
received by the crews exposed to each display condition. The number of resolution advisories that 
would have been received by the control crews had they had TCAS is also shown, for comparison. 
Chi-square analysis indicated that there was not a significant difference between control and TCAS 
crews with regard to the frequency of traffic conflicts. 

A total of 78 resolution advisories were of the following types: 

Climb 18 (23%) 

Do not climb 8 (10%) 

Descend 18 (23%) 

Do not descend 34 (44%) 

Total 78 (100%) 

Maneuver advisories with visual contact totaled 55% ; preventive advisories (requiring no maneuver) 
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totaled 24% ; and altitude-crossing maneuver advisories totaled 19% . 

Note that the frequency of advisories simulated in this study is far greater than that measured in air- 
craft operating with TCAS (table 5). The rate of traffic and maneuver advisories was established arbi- 
trarily to provide as much pilot performance data as possible within the confines of the single day 
available for each experimental flight crew, without unduly burdening them or making their situation 
grossly unrealistic. An effort was made to present the conflicts at irregular intervals and during por- 
tions of the flights representing both high and low workload periods. 


TABLE 5. TCAS ALERTS OBSERVED DURING LINE FLYING 


Flight 

condition 

Traffic 

advisories 

Resolution 

advisories 

Ratio of 
TA : RA 

828 hrs of line flying 

473 

38 

12.4 : 1 

Number per flight hr 

0.57 

0.05 

— 

Maneuver advisories with visual contact 
Preventive advisories 

(Womack 1987) 

73% 

16% 



RESULTS: ORGANIZED BY RESEARCH QUESTIONS 

The research questions addressed in the data analysis and the results of the analyses are described in 
the following sections. A summary of this information can be found in the Executive Summary. 


1. Did TCAS improve safety? 


The primary measure of TCAS effectiveness is that some reasonable standard of separation is main- 
tained between conflicting aircraft. In this study, horizontal separations of less than 500 and 1000 ft, 
and vertical separations of less than 100 and 200 ft, were arbitrarily chosen as criteria for separation. 


When TCAS was in use, there were no occurrences in which minimum separation from transponder- 
equipped intruder aircraft was ever less than 1000 ft horizontally and 200 ft vertically at the same ume. 

Crews not using TCAS were exposed to the same traffic conflicts (though random variation in flight 
paths produced variation in precise encounter geometries). There was one instance of separation of 
less than 500 ft horizontally and 100 ft vertically. The crew was in visual contact with this aircraft, 
which was not announced by ATC, for 13 sec before it passed 316 ft horizontally and 32 ft vertic y 
from their aircraft. In addition, there were three instances of separation less than 1000 ft honzontaiiy 
and 200 ft vertically; all of these aircraft were visually acquired at some time during the encounter. 
Minimum separation at the point of closest approach was 529 ft horizontally and 58 ft vertically in this 
group. The crews using TCAS were significantly less likely to incur inadequate separation by this cri- 
terion (less than 1000 ft horizontally and 200 ft vertically) (exact binomial test, p<.05). 
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For TCAS cases, the separation gained as a result of avoidance maneuvers was determined by extrapo- 
lating from the aircraft’s flight path prior to the maneuver to the closest point of approach. In 37 of 40 
cases examined in this way, adherence to the TCAS maneuver command produced increased separation 
between conflicting aircraft. In four cases, separation without the maneuver would have been less than 
1000/200 ft; in one of these, minimum separation would have been 402 and 18ft. The methodology 
and results from this analysis are described in Appendix Q and (Tuttell, 1988). In 3 of 40 cases, 
separation associated with adherence to the TCAS-commanded maneuver was slightly decreased by the 
commanded maneuver. Analysis (Lubkowski, D., MITRE Corporation, McLean, Virginia. Personal 
Communication, March, 1988.) indicated that TCAS inaccurately predicted the time to CPA because of 
a large lateral miss distance. The TCAS II logic does not consider bearing information; the system 
was therefore unaware that the lateral miss distance would be so large. In none of these three 
encounters was closest separation less than 9000 ft horizontally. 

2. Are pilots able to initiate avoidance maneuvers within the 5 sec allocated by the TCAS logic? 

TCAS is designed to evaluate conflict aircraft trajectories long enough to minimize the number of false 
alarms while allowing ample time for pilot and aircraft response to provide adequate separation. The 
mean response time found in this simulation was 1.9 sec from the onset of a corrective resolution 
advisory to the first fore/aft stick or throttle movement (fig. 2). The standard deviation was 1.34 sec. 
There was only one instance in the 57 advisories requiring a maneuver in which a pilot responded 
more slowly than 5 sec, i.e., in 7 sec. This did not result in unsafe separation. 
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SECONDS 

HE AN = 1.912 ST DEU= 1.340 N= 57 

Figure 2. Distribution of initial response times to TCAS corrective resolution advisories. These 
times were evaluated from digital printouts of altitude, altitude rate, throttle cross-shaft angle 
and control column position evaluated once each sec. 
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The reaction times found in a study such as this may well be better than in operational service, since 
the pilots knew that TCAS was under investigation. Nonetheless, this study does demonstrate that the 
range of reaction times is within the limits of the system. 

3. Do pilots respond correctly to the maneuver advisories presented on the vertical speed indicator 
and annunciated in the verbal warnings? 

All responses in this study were in the correct direction (e.g., all pilots descended when the system 
commanded a descent). This differs from results obtained by Boucek, et al„ 1985. The pilots 
responded with greater rates of climb and descent than were required by the system (see question 4). 
This caused separation to increase more rapidly, but altitude changes were greater than were required. 
The degree of response accuracy was measured by the root mean square of the difference between the 
vertical velocity and the commanded vertical velocity. RMS overshoots ranged from 48 to 5517 
ft/min, with a mean of 1327, and standard deviation of 1152. 

4. Using TCAS, are pilots able to attain safe separation with minimal impact on air traffic control? 

The not unreasonable concern has been expressed that pilots responding to TCAS commands may 
maneuver into the altitude of another aircraft. This is most likely in low altitude high density terminal 
airspace where vertical separation is 1000 ft. The mean altitude change from level flight required by 
TCAS during corrective maneuver advisories in this experiment was 652 ft, with a range from 16 to 
1971 ft. The mean altitude change from level flight as a result of pilot responses to corrective 
maneuver advisories was 852 ft, with a range from 16 to 2123 ft, and a standard deviation of 583 ft 
(n=20) (fig. 3). 

Nine pilots responded actively to a preventive resolution advisory, in which a maneuver was not 
required. The mean altitude change from level flight as a result of pilot responses to preventive 
maneuver advisories was 206 ft, with a range from 3 to 854 ft, and a standard deviation of 309 ft 
( n =9) (fig. 4). The mean altitude change from level flight for both corrective and preventive advisories 
was 652 ft, (range 3 to 2123, standard deviation 592, n = 29). 

5. Did pilots with more traffic information respond differently to maneuver advisories? 

Four captains and four first officers were exposed to each level of traffic informatioa The number of 
maneuvers made with each of these traffic displays ranged from 21 to 33, averaging 26 per display 
condition. Analyses of variance were performed on the pilot performance measures by display condi- 
tion. These included (1) response time from the onset of the maneuver advisory to the initiation of 
stick or throttle movement to change the vertical speed, (2) time to attain commanded vertical speed, 
(3) accuracy of vertical speed (root mean square (rms) of vertical velocity error and peak vertical velo- 
city overshoot), (4) time to initiate a return to altitude after the traffic conflict, and (5) magnitude of 
altitude change as a result of the maneuver. 

No differences in response time were found as a function of the amount of traffic information 
presented. Only peak vertical velocity overshoot showed significant differences across levels of traffic 
information (F=5.28, df=2,64, p<.01). Crews with no traffic display had the greatest peak overshoot 
(mean of 2272 ft/min greater than commanded velocity). Crews with a continuous traffic display of all 
traffic had the next highest peak overshoot (1317). The crews with traffic displayed only dunng a 
conflict had the least overshoot with a mean of 1221 ft/min. This measure is only a momentary value 
and does not represent performance throughout the avoidance maneuver; the rms vertical velocity 
overshoot is a continuous measure of the accuracy of the avoidance maneuver, and this did not differ 
significantly across conditions. Changes in altitude during avoidance maneuvers did not vary with 
display condition. 
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Figure 8. Altitude changes from level flight in response to corrective resolution advisories. 
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Figure 4. Altitude changes from level flight following preventive resolution advisories. 
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Pilots using TCAS (conditions 2, 3, and 4) were asked to rate the usefulness of the traffic advisory in 
preparing for evasive maneuvers on a scale from 1 to 10. For those with no traffic display, this meant 
only a light and a tone. There were no differences in ratings associated with the level of traffic infor- 
mation provided. The mean was 7.7, with a standard deviation of 1.4. 

The level of traffic information was associated with differences in some ratings, however. Distraction 
perceived as caused by the addition of TCAS to other flight duties was generally rated as low 
(mean=2.6). The distraction rating was least for those with traffic display during a conflict only 
(mean=1.8, median=1.5); those with no traffic display had a mean of 2.1 (median=1.5). Those with 
continuous traffic information rated the distraction as 3.8 (median=3.5). These differences were 
significant (F=3.62, df=2,33, p<.05, for the arcsin transformed data). 

The same pattern was found when crew members were asked to rate the distraction of the auditory 
alert and caution/waming lights in their function of attracting the crew’s attention. All TCAS crews 
had the same auditory alert and caution/waming lights, yet there were differences in their perceptions 
of the distraction caused by these alerts (F=6.15, df=2,33, p<.01, for the arcsin transformed data). 
Crews with no traffic display rated the distraction 2.2 (median=1.5). Those with traffic information 
during a conflict only gave a mean rating of 1.5 (median=1.0). Pilots with continuous traffic informa- 
tion rated distraction as 4.4 (median of 4.5) on a 10-point scale. 

Crews with no traffic display (condition 2) rated the perceived effectiveness of the auditory alert and 
caution/waming light in getting the pilot’s attention as 8.5 on a ten-point scale (median=9.0). Pilots 
with traffic during conflicts only (condition 3) gave mean and median ratings of 8.0. Pilots with con- 
tinuous traffic information rated the distraction of the auditory alert and lights highest, but rated the 
effectiveness lowest (mean=7.2, median=8.0) (F=3.44, df=2,33, p<.05 for the arcsin transformed data). 


6. Are pilots able to use TCAS correctly without a traffic display? 

In condition 2, pilots flew without a planform display of traffic. Their responses to maneuver 
advisories did not differ in magnitude, or kind, from those of the crews that saw displays of the 
conflicting traffic. Performance by crews having no traffic display was equivalent to that of crews who 
had displays of traffic either during conflicts or continuously. The only measure that differed 
significantly when these two groups were compared was peak vertical velocity. As noted above, pilots 
without a display of traffic information had higher peak vertical velocity overshoots (mean of 2272 
ft/min greater than commanded) during the performance of the TCAS maneuvers than those who were 
able to see the location of conflicting traffic on their displays prior to the maneuver (1277 ft/min), 
(F=10.62, df=l,65, p<.01). RMS velocity overshoot however, did not differ among groups. 

The pilots who had a traffic display were asked "Did the traffic advisory display provide confidence in 
the correctness of the subsequent maneuver advisory?" All 24 agreed that it did. 

7. Did pilots respond differently to different maneuver advisories ? 

The maneuvers were grouped into: climb, descend, reduce rate of climb, and reduce rate of descent 
Reaction time to the advisory, response accuracy, and time to initiate a return to altitude did not vary 
with the type of maneuver. 

The period from the time of the resolution advisory to the attainment of commanded vertical velocity 
was least when reducing rate of descent (4.5 sec from onset of advisory), only slightly greater for 
climbs (4.7), descents were next at 6.2, and it took pilots longest to attain the commanded vertical 
speed when reducing rate of climb (7.8 sec) (F=4.03, df=3,53, p<.05). The altitude change as a result 
of the maneuver was least for the advisory to reduce climb (153 ft), next were reductions in descent 
rate (311 ft), climbs averaged 725, and descents 790 ft (F=6.84, df=3,54, p<.001). There were no 
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significant interactions between level of traffic information and maneuver type. 


8. Did pilots alter their flight path in response to a preventive advisory? 

At least 30% of TCAS resolution advisories do not require a change in flight path. Such alerts indi- 
cate that the present flight path is safe, but the pilot is advised of a range of vertical speeds that would 
take his aircraft too close to the intruder. In this experiment these advisories were issued as cauuons 
and were not accompanied by a voice warning. There were 19 preventive advisories, 24 percent of the 
total On 9 occasions, crews did not follow the instructions in the training program and climbed or 
descended from level flight when they received a preventive advisory. The maximum vertical velocity 
was +2081 ft/min in response to a preventive "do not descend" advisory which occurred while the ^air- 
craft was in level flight on autopilot For those cases in which a change of altitude was miuated from 
level flight the mean altitude change was 206 ft, with a range from 3 to 854 ft, and a standard devia- 
tion of 309 ft (fig. 4). 


9. Were pilots more likely to visually acquire an intruder with more traffic information? 

The probability of visual detection of traffic that had evoked a TCAS traffic advisory (or that would 
have done so, in the case of condition 1) was virtually the same for all conditions, about 55% (table 
6). Similarly, the traffic recorded by observers as not detected, based on a statement from within the 
cockpit, did not differ significantly across conditions (chi-square = 7.8, d.f. = 9, n.s.). 

TABLE 6. VISUAL DETECTION OF TRAFFIC vs LEVEL OF INFORMATION 


Experiment 

condition 

Total traffic 
advisories 

Detected 

visually 

Not detected 
visually 

Unknown 

1. No TCAS 

84 

44 (52%) 

25 (30%) 

15 (18%) 

2. No traffic display 

67 

39 (58%) 

25 (37%) 

3 ( 5%) 

3. Conflict display 

64 

36 (56%) 

17 (27%) 

11 (17%) 

4. Continuous display 

76 

41 (54%) 

26 (34%) 

9 (12%) 


One non-transponder target was presented; it crossed the 727’s flight path near final approach altitude 
just outside the outer matter under conditions of limited visibility. This was a difficult target to see 
and was designed to evaluate outside visual scan at a busy time in the flight The target was detected 
by three of four non-TCAS crews (75%) and by 5 of 12 TCAS crews (42%); but because of the smaU 
number of observations, the difference, though suggestive, is not statistically significant. It cannot be 
said with certainty, therefore, either that visual detection of intruders was enhanced by the presence of 
TCAS, with or without a planform display, or that outside visual scan was less comprehensive and 
effective because of the presence of TCAS within the cockpit. 


A degree of caution must be exercised in extrapolating these results to the real world. The simulator 
visual system’s portrayal of other air traffic is accurate with respect to location, but other aircraft fight- 
ing is less bright than in the real world. All other things being equal, one would expect a higher pro- 
bability of detection in actual flight than in the simulator. 


might be expected, the levels of traffic information provided were reflected in the crews percep- 
of the assistance provided by TCAS in visual detections of traffic. The crews were asked to rate 
useful TCAS was in this regard. On a 10-point scale from "not at all useful to very useful, 
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crews with no traffic display gave a mean rating of 4.3 (median 4.5); those with traffic presented only 
during conflicts rated the usefulness at 8.3 (median 9.0); and crews with continuous traffic information 
rated TCAS usefulness at 9.25 (median 9.5) (F=21.26, df=2,33, p<.001 for arcsin-transformed rating 
data). The difference between the perceived and observed usefulness of the traffic display for visual 
detection was substantial in this study. 

10. Did pilots hesitate or respond differently when a maneuver advisory occurred after visually acquir- 
ing an intruder? 

A one-way analysis of variance showed no performance effects associated with pilots having visual 
contact with the conflicting aircraft (See the discussion in question 15 regarding visual sighting in the 
altitude crossing case.) 

11. Did the level of traffic information affect pilots’ responses differently when a maneuver advisory 
occurred after visually acquiring an intruder? 

The only performance measure which showed an interaction between display condition and visual 
acquisition was time to attain commanded vertical velocity (F=4.32, df=2,51, p<.05) (table 7). The 
time to establish the vertical velocity prescribed by TCAS was greater when the crews had the traffic 
in sight, except for those crews who had a continuous display of all the traffic. 


TABLE 7. MEAN TIME TO ATTAIN COMMANDED VERTICAL VELOCITY 



Condition 2 

Condition 3 

Condition 4 

Visual 

no traffic 

conflict traffic 

continuous traffic 

acquisition 

(sec) 

(sec) 

(sec) 

Acquired 

4.87 

7.25 

5.09 

Not acquired 

4.75 

3.67 

6.08 


12. Did pilots respond differently to a maneuver advisory which occurred in instrument meteorological 
conditions (1MC)? 

Since pilots flying under EMC were not able to visually clear the airspace into which they were 
maneuvering, it was speculated that they might hesitate in their responses. There were no statistically 
significant differences between the performance of maneuvers in instrument and in visual conditions. 

13. Was any hesitation due to instrument meteorological conditions affected by the amount of traffic 
information? 

There were no such interactions for any of the performance measures. 

Pilots were asked "Were you reluctant to perform a TCAS maneuver in instrument meteorological con- 
ditions?". They gave ratings from "not at all reluctant" to "very reluctant" on a 10-point scale. Their 
responses differed as a function of the amount of traffic information available (F=4.72, df=2,31, p<.05 
for arcsin transformed rating data). Those with traffic displayed only during a conflict were least reluc- 
tant (mean=2.5, median=2.0); those with no traffic display were virtually equal in their reluctance 
(mean=2.6, median=2.0). The pilots with continuous traffic position information perceived themselves 
as most reluctant to maneuver in instrument meteorological conditions (mean=5.2, median=6.0). Their 
performance, however, did not mirror these subjective ratings. 
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14. Did pilots hesitate or respond incorrectly when a maneuver advisory required crossing through 
another aircraft's altitude? 

If a conflicting aircraft is climbing or descending at a rapid rate, TCAS may issue a maneuver advisory 
that requires the pilot to cross through the other aircraft’s present altitude. For example, if the intruder 
is above the TCAS aircraft but descending, the best maneuver may be a climb, since the other aircraft 
will be below ownship at the point of closest approach. There were significant differences in pilot per- 
formance between those maneuvers which required pilots to cross through the intruder aircraft’s alti- 
tude and those that did not 

A one-way analysis of variance of reaction times showed significantly longer times from the onset of 
the advisory to the first stick or throttle movement when the advisory required passing through an 
intruder’s altitude (F=10.63, df=l,63, pc.Ol). The mean response time when crossing altitudes was 2.9 
sec, vs 1.7 sec when not crossing altitudes. Pilots may have compensated for the longer response time 
with a higher vertical speed. The rms vertical velocity was greater than required by 2163 ft/min when 
the pilots were climbing or descending through the intruder s altitude vs 1081 ftAnin when not crossing 
(F=10 33 df=l,55, pc.Ol). The peak overshoot in vertical velocity was also greater when crossing 
altitudes, 2447 vs 1346 ft/min (F=9.92, df=l,65, pc.Ol). The mean time to return to the altitude at 
which the maneuver began was also longer when crossing altitudes, 42.4 vs 19.5 sec when not cross- 
ing, (F= 13.34, df=l,46, pc.001). As would be expected from these results, the amount of altitude 
change was greater as a result of a maneuver crossing through the other aircraft’s altitude, 1 100 vs 379 
ft (F=37.69, df=l,56, pc.001). 

15. Did visual acquisition affect pilot performance during an altitude crossing maneuver? 

There were no differences in performance as a function of visual acquisition when the maneuver 
required that pilots cross through an intruder’s altitude. That is, no performance measure showed 
significant interactions between visual acquisition and altitude crossing encounters. 

There was one instance in which a crew correctly anticipated an altitude crossing maneuver by visually 
monitoring the intruder’s flight path. They maneuvered prior to the maneuver advisory and descended 
through the other aircraft’s altitude. In another instance, however, a captain instructed the first officer 
not to follow a climb maneuver advisory because he saw the traffic above. He then realized that the 
other aircraft was descending, and they indeed needed to climb to avoid it. At that moment he took 
control of the aircraft and executed, the climb maneuver. 


16. Did pilot responses differ as a function of more traffic information when the evasive maneuver 
required crossing through the intruder's altitude? 

Both the level of traffic information and altitude crossing maneuvers were significant in a two-way 
analysis of variance of the time to initiate a response (F=6.02, df=2,59, pc.Ol and F=12.85, df=l,59, 
pc.001). Pilots with no traffic display were quickest to initiate the maneuver with a mean reaction 
time of 1.6 sec; those with traffic always present had a mean of 1.8, and those with traffic only during 
a conflict averaged 2.6 sec. As discussed above, the time to initiate a maneuver was greater when the 
maneuver would take the aircraft through the altitude of the traffic. The interaction between level of 
traffic information and altitude crossing/noncrossing maneuvers was significant for reaction time 
(F=3.47, df=2,59, pc.05). The difference between crossing and noncrossing reaction times became 
greater as the mean reaction time increased across traffic display conditions (table 8). 
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TABLE 8. RESPONSE TIMES vs LEVEL OF INFORMATION AND 
CROSSING REQUIREMENT 


Condition 2 Condition 3 Condition 4 
Required no traffic conflict traffic continuous traffic 

maneuver ('sec) fsec) (sec) 

Altitude crossing 1.60 4.25 3.00 

No crossing 1.59 2.00 1.50 


The interaction between level of traffic information and altitude crossing/noncrossing was also 
significant for rms vertical velocity error (F=5.07, df=2,51, p<.01). Those with only conflict traffic 
displayed had the greatest rms vertical velocity when crossing and the least when not crossing the 
intruder’s altitude (table 9). 


TABLE 9. RMS VERTICAL VELOCITY ERROR vs CROSSING MANEUVERS 


Required 

maneuver 

Condition 2 
no traffic 
(ft) 

Condition 3 
conflict traffic 
fft) 

Condition 4 
continuous traffic 
(ft) 

Altitude crossing 
No crossing 

1812 

1554 

3243 

697 

1520 

1016 


Pilots were asked "were you reluctant to perform a TCAS maneuver that required you to cross through 
another aircraft’s altitude?". They rated their answers from "not at all reluctant" to "very reluctant" on 
a ten-point scale. Their responses differed based on the amount of traffic information they had avail- 
able (F=4 24, df=2,26, p<.05 for arcsin transformed rating data). Those with the traffic display only 
during a conflict were least reluctant (mean=3.1, median=2.0), and those with no traffic display were 
next in their reluctance (mean=4.6, median=5.0). The pilots with continuous traffic position informa- 
tion were the most reluctant to cross through an intruder’s altitude (mean=5.5, median=6.0). 

17. Was the response to a maneuver advisory affected by the aircraft’ s flight attitude? Did this vary 
with the amount of traffic information? 

The average rim<» from the onset of the maneuver advisory to the first fore/aft stick or throttle move- 
ment was greater when maneuvering from level (2.5 sec) than from a climb or descent (1.5 sec) 
(F=9.97, df=l,63, p<.01). The rms vertical velocity overshoot was also greater when maneuvering 
from level (1695 ft/min) than from a climb or descent (1079) (F=4. 14, df=l,55, p<.05). The altitude 
change as a result of the maneuver was greater when maneuvering from level (652 ft) than from a 
climb or descent (404 ft) (F=4.34, df=l,56, p<.05). The time to return to within 300 ft of the altitude 
at which the maneuver began was compared to the time to reestablish the rate of climb/descent from 
which the maneuver began. These were significantly different (F=34.86, df=l,46, and p<.001). When 
maneuvering from level flight, the mean time to return to an altitude was 40.8 sec, with a standard 
deviation of 23.1 sec. The time to reestablish the rate of climb/descent averaged 13.1 sec and had a 
standard deviation of 7.0 sec. There were no interactions between aircraft flight attitude and the level 
of traffic information for any of the performance measures. 
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18. Did pilot reaction time to the maneuver advisory or response accuracy improve with experience or 
deteriorate with fatigue? Was pilot performance affected differently by experience, given different 
levels of traffic information? 

There was no change over time in any of the performance measures, on average. Had a learning effect 
been present, one might have anticipated improvement in these measures. Fatigue, on the other hand, 
might have been expected to produce progressive decrements in performance (duty times often 
exceeded 10 hours). It is not possible to separate one effect from the other, all that can be said is that 
no trends in either direction were found in the performance data. 

There were no interactions for any performance measure between the level of traffic information and 
the sequential number of the maneuver. 

19. Were there any effects on workload when TCAS was added to the cockpit? Did increasing the 
amount of traffic information effect crew workload? 

Participants in the experiment were briefed during their instruction period regarding the use of the 
NASA Task Load Index (TLX) instrument (appendix N). During the experiment, each flight crew 
member filled out a brief workload questionnaire at the conclusion of each flight segment. The work- 
load scales evaluate perceived the mental demand, physical demand, temporal demand, performance, 
effort, and frustration. The workload instrument has been validated in previous simulation and flight 
research (Hart & Staveland; Shively et al., 1987). 

Overall workload was significantly higher in captains and first officers when they were flying the air- 
craft. There were significant differences among flight segments, shorter segments and segments into 
high-density terminal areas being associated with higher workload. Second officer workload ratings 
showed a strong association with flight segments. 

Second officers tended to perceive higher workload without TCAS than with it, but the trend was not 
significant. First officers had nearly significant differences in perceived workload (p=.058) across 
TCAS conditions; condition 3 (traffic display only during conflicts) invoked the lowest level of work- 
load and condition 4 (full-time traffic display) the highest. Captains perceived no significant difference 
in workload across information levels (Battiste, V; and Bortolussi, M. R.: NASA Ames Research 
Center report in preparation, 1988). 

20. What did pilots do when a TCAS invalid occurred? 

The training program advised the crews to resolve the conflict themselves using all available informa- 
tion Also they were advised to stop following the previous TCAS advisory and return to previous 
clearance, if notified of a TCAS invalid (i.e., when the present maneuver was not projected to provide 
at least 100 ft vertical separation in the correct direction at CPA). Despite the inclusion in the experi- 
ment of a double conflict, designed in an attempt to invoke an invalid for each flight crew, only one 
TCAS invalid in a TCAS-equipped flight occurred. 

In the single TCAS Invalid occurrence, a descent maneuver advisory was issued; this was followed 5 
sec later by an invalid advisory, "unable to command." The crew began to arrest their descent rate. 
They received a climb advisory to avoid the other aircraft 4 sec after the invalid. The crew responded 
to the climb advisory, and they received a second invalid 7 sec after the climb advisory. They ignored 
this advisory and continued to climb. The traffic was visually acquired. 
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21 . Did pilots maneuver prior to a maneuver advisory based on the information on the traffic display? 
If so, were they more likely to do so with more traffic information? 

The TCAS traffic advisory display is designed to assist pilots to establish visual contact with 
conflicting traffic. It may also be used to observe the flight paths of nearby traffic and to monitor the 
relative altitude differences between the TCAS aircraft and other aircraft in the vicinity. This allows 
the pilots to see dangerous situations developing and to prepare for possible evasive maneuvering 
directed by TCAS. 

Eight of the 16 flight crews flew with this traffic display, shown either during conflicts or continuously. 
They were thoroughly briefed that the traffic advisory display was for traffic information only and was 
not to be used for evasive maneuvering. The IVSI display was to be used for evasive maneuvers in 
the vertical plane following a maneuver advisory. In general, the pilots adhered to the guidelines that 
they were given. 

There were 14 detected occurrences, however, in which the pilots used their own experience and judg- 
ment to maneuver the aircraft based on the traffic advisory display information. Each of these incidents 
is described in detail in appendix R and Tuttell, 1988. A summary follows. 

Course and altitude adjustments by crews with a traffic display showing the location of traffic only 
during a conflict (condition 3): 


Crew 1: 
Crew 2: 
Crew 3: 
Crew 4: 


No maneuvers 

2 altitude adjustments and 2 turns 
1 altitude adjustment 
1 altitude adjustment and 1 turn 


Course and altitude adjustments by crews with a traffic display showing the location of traffic continu- 
ously (condition 4): 


Crew 1: 1 altitude adjustment 

Crew 2: 3 altitude adjustments 

Crew 3: No altitude adjustments; 1 turn 

Crew 4: 1 altitude adjustment and 1 turn 

An analysis of the fourteen incidents shows a few patterns. Altitude adjustments accounted for 64% of 
the total adjustments (9 out of 14). The majority of the adjustments occurred during descents (10 out 
of 14). Three of the turn adjustments involved maneuvers to avoid a mode A (no altitude reporting) 
transponder equipped aircraft The most common scenario involved the TCAS aircraft descending on 
top of another aircraft These situations gave the pilots enough warning so that they could observe the 
dangerous situation developing on the traffic advisory display and take corrective action. The correc- 
tive action usually resulted in a decrease in the rate of descent or a level-off above the assigned alti- 
tude for a short period of time. There were no altitude changes from level flight. All of the fourteen 
adjustments caused small deviations from air traffic control clearances for short time periods. Each 
crew attempted to notify air traffic control of the altitude deviations as soon as workload permitted. 

As shown above, there were seven cases of maneuvers prior to a maneuver advisory based on traffic 
location information by pilots who had the traffic presented only during a conflict, and an equal 
number of anticipatory maneuvers by crews who had traffic information continuously presented. 

Crew members were asked "Do you feel the information on the traffic display is accurate enough to be 
used to maneuver your aircraft?" Twelve flew with displays with a 2° rms error in the relative bearing 
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of the traffic. Eleven of them answered yes to the above question. Eleven of the twelve pilots who 
flew with displays having an 8° rms error also felt the information was accurate enough to be used for 
maneuvering. 


22. How did evasive action differ with and without TCAS? 


As noted above, 9 of 14 anticipatory maneuvers by TCAS-equipped crews were altitude or altitude rate 
adjustments, five were turns. Non-TCAS crews (condition 1) maneuvered on 17 visually sighted 
intruders that would have caused maneuver advisories, had they been TCAS-equipped. Of these 
maneuvers, 9 were altitude or altitude rate adjustments, 8 were turns. 


The pilots in condition 1 generally made less abrupt maneuvers than did pilots using TCAS, probably 
because they maneuvered only enough to maintain visual separation rather than observing the set 
response pattern required by TCAS procedures. A few maneuvers were severe, however, one involved 
a peak descent rate of 7832 ft/min, in a descent case which may have involved a visual misperception 
resulting from aircraft pitch angle at the time of the encounter. 

The hypothesized visual misperception was noted several times during test flying of these scenarios. It 
occurs by virtue of the substantial range of pitch angles attained by turbojet aircraft during normal 
operations, approximately +20 to -ltf> . In a climb or descent at night, without a visible horizon and 
especially without ground lights, it can be very difficult to perceive whether a conflicting aircraft is 
above or below ownship, and whether it is climbing, stationary, or descending with respect to ownship. 
TCAS, and especially its traffic display, can be extremely helpful by resolving the perceptual ambiguity 
inherent in such situations. 


23. What individual differences were there in pilots' response to maneuver advisories? 

One of the more interesting findings of this study was the variability in responses to TCAS avoidance 
maneuvers. The range of reaction times was from 1 to 7 sec; the standard deviation was 1.3 sec 
(n=57) The range of times to attain the commanded vertical speed was from 1 to 10 sec; the stan- 
dard deviation was 2.2 sec (n=54). The range of rms vertical velocity overshoots was from 182 to 
5517 ft/min; the standard deviation was 1172 ft/min (n=52). The range of times to initiate a retumto 
altitude when clear of the traffic was from 1 to 6 sec; the standard deviation was 1.8 sec (n-44). The 
time to return to the altitude at which the aircraft was level at the start of the maneuver, or, if not 
level, the time to reestablish the previous rate of climb or descent, ranged from 2 to 93 sec, with a 
standard deviation of 20.9 sec. 


Pilots were asked to evaluate their understanding of the operation and limitations of TCAS from very 
limited to very complete, 1 to 10. The range of responses was 3 to 9, with a standard deviation of 1.4 
(n=36). Klots were also asked to rate the operational procedures for TCAS from very inappropnate to 
very appropriate. The range of responses was from 5 to 9, with a standard deviation of 1.1 (n-36). 
They were asked the question "Would you be satisfied with this TCAS system as a safety enhancement 
to the airplane you fly?" They rated the system from "greatly dissatisfied' to greatly satisfied. The 
responses ranged from 3 to 9, with a standard deviation of 1.4 (n=36). There were no significant 
differences associated with information level. 

24. Were there performance differences between captains and first officers? 

There were no differences between the responses to maneuver advisories by captains and first officers. 
There were also no interactions between level of traffic information and crew position for any o e 
performance measures. None of the questions on the questionnaire were answered differently by cap- 
tains and first officers. The total flight time for captains averaged 13,250 hr with a range of 3500 to 
21,000. The pilot with 3500 hr was currently flying as first officer for his airiine but had begun 
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training for captain. The mean total flight time for first officers was 7100 with a range of 3500 to 
12,000 hr. 

25. Did pilots assigned to the display conditions differ inflight experience? 

The crews were randomly assigned to the display conditions. An analysis of differences between con- 
ditions vs piloting experience showed no differences in age, total flying time, or time in crew position. 
There were significant differences in Boeing 727 flight time among the pilots assigned to the display 
conditions. Pilots with no traffic display had a mean of 2008 hr, those with traffic displayed only dur- 
ing a conflict averaged 4379, and the pilots with traffic continuously presented had 1182 hr of Boeing 
727 time (F=5.70, df=2,32, pc.Ol). Flight time in the last 90 days also differed among pilots across 
the three display conditions: those with no display averaged 136 hr, those with only conflict traffic 
displayed, 204, and those with all the traffic displayed, 129 hr (F=5.29, df=2,32, p<.05). 

26. Did the pilots assigned to each display condition evaluate the simulation equally? 

There were no differences among the pilots assigned to the four display conditions in their evaluation 
of the simulation. Questions 1-4 in appendix O addressed the procedures, the air traffic control ser- 
vices, the handling qualities of the simulator, and the differences due to airline configuration. The 
pilots in all the display conditions rated these qualities similarly. See appendix S for the responses to 
each question. 

27. What were the pilots’ subjective evaluations of the TCAS? 

Pilots rated the adequacy of the TCAS training they received as 8.3 on a scale of 1 to 10 (from "not at 
all adequate" to "very adequate"). Their mean rating of the distraction due to the addition of TCAS to 
their other flight duties was 2.6. (See question 5.) TCAS was rated as useful for reducing the risk of 
midair collisions (mean=8.0). TCAS was rated as useful in aiding visual contact (mean=7.3). (See 
question 9.) 

The evasive maneuvers prescribed by TCAS were rated 7.7 on a 10-point scale from unacceptable to 
acceptable. Pilots rated their satisfaction with the TCAS as 7.6. See appendix S for the mean pilot 
ratings of the TCAS in general, its use, and the various components of the system. 

28. Were pilots accurate in the estimate of their reaction time to the TCAS maneuver advisory? 

Pilots were asked to estimate their reaction time in responding to a maneuver advisory. The average 
estimated time to initiate a maneuver was 3.2 sec. The measured mean reaction time was 1.9 sec, an 
average difference of 1.3 sec. The differences between the estimated and actual reaction times differed 
significantly (T=2.42, df=44, p<.05) and were not correlated. 

29. With traffic automatically presented only during a conflict, how often did pilots use the traffic 
switch to enable the display? 

The pilots who were presented traffic location information only during a conflict were able to show the 
three top priority aircraft at any time by toggling a switch on the transponder/TCAS panel. Crews fre- 
quently used this function after receiving an ATC traffic advisory, as an instance. The display would 
become active for 15 sec each time this switch was selected. Sometimes crews would display the 
traffic for 15 sec and immediately display it again. When the display was enabled sequentially in this 
manner, it was counted as a multiple use. Crew 1 displayed the traffic 54 times with nine multiples. 
Crew 2 used this display option 19 times including two sequential selections. The third crew used the 
traffic display selector 16 times. Twice, there were double selections. Crew 4 toggled the display 
switch the most, with 91 display presentations and 17 multiples. 
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The pilots who had traffic presented only during a traffic conflict were asked to rate the usefulness of 
the option to display the most important targets. The mean rating was 7.8, with a standard deviation of 
2.1 on a 10-point scale. 

30. How did pilots having continuously displayed traffic information use the options for the traffic 
display ranges? 

The four crews having continuous traffic information were able to select the range of traffic to be 
displayed to either 3, 5, 10, or 20 nautical miles. Table 10 shows the usage of this range option. 

They could also select the vertical range for up 7000 and down 2700 ft (above), up and down 2700 
(off), or up 2700 and down 7000 ft (below). Table 11 shows the ranges selected by the four crews. 
For a description of the suggested use of these options see the training video narrative (appendix D). 

TABLE 10. USE OF THE HORIZONTAL DISPLAY RANGE OPTIONS 
(Data for four crews, approximately 6-hr flight time each) 


Range 

setting 

Number 
of times 
selected 

Average 
duration 
at each 
range 

(min:secl_ 

Maximum 
duration 
of range 
selection 
(min:secl 

Average 
altitude 
at time of 
selection 

Percentage 
of total 
time 

3 

11 

1:05 

5:46 

10400 

1.1 

5 

43 

2:38 

24:42 

9640 

10.8 

10 

80 

5:43 

36:24 

11136 

44.8 

20 

58 

7:40 

49:36 

12388 

43.3 


TABLE 11. USE OF THE VERTICAL RANGE OPTIONS 


Vertical 

setting 

Number 
of times 
selected 

Average 
duration 
at each 
range 
( min: seel 

Maximum 
duration 
of range 
selection 
fhr:min:secl_ 

Average 
altitude 
at time of 
selection 
(ft) 

Percentage 
of total 
time 

Above 

50 

1:52 

0:17:25 

17449 

12.7 

Below 

52 

1:27 

0:10:56 

21261 

9.4 

Off 

71 

8:40 

1:43:57 

18619 

77.9 


The flight level switch converted the relative altitudes of the traffic to their altitude above sea level. 
This option was selected 7 times by the 4 crews, averaging 1.75 times per crew. Crew 1 selected the 
flight level switch 0 times, crew 2 once, crew 3 twice, and crew 4 four times. 
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CONCLUDING REMARKS 


It is known from many simulation experiments, and from the use of full-mission simulation for line- 
oriented flight training, that pilots (and especially air carrier pilots) may behave differently under simu- 
lated flight conditions than they would in the real world. It is also known, however, that if efforts are 
made to construct a simulation realistically, the behavior of most pilots approaches their usual behavior 
in actual flight operations. This is most likely to be true if crew members perceive themselves as not 
under direct or continuous observation. 

In this simulation experiment, everything possible was done to minimize the intrusiveness of the exper- 
imental situation. No observers were ever present in the cockpit; interactions with flight crews were 
only those normal to line flying. Every effort was made, on the other hand, to provide crews with all 
of the interactions with ATC, company, and ground crews that they normally would encounter during 
routine operations. Based on crew comments during debriefing, these efforts were largely effective. 

Nonetheless, the results of this experiment must be interpreted cautiously. While there were periods of 
over an hour without traffic conflicts, the crews were exposed to 6 times as many traffic advisories and 
over 20 times as many maneuver advisories as have been experienced in line flying. They knew that 
TCAS, and specifically their responses to the system, were under investigation, and they knew, of 
course, that conflicting traffic posed no actual danger to them. 

Bearing in mind these limitations of this (or any other) simulation experiment, the authors believe that 
certain observations can be made and certain conclusions drawn from the data collected during this 
study. 


SUMMARY OF RESULTS 

Within the conditions of this experiment, the TCAS II traffic alert and collision-avoidance system, as 
implemented, was entirely successful in ameliorating the seriousness of the traffic conflicts presented to 
flight crews. The twelve crews flying with TCAS had no conflicts involving a point of closest 
approach of less than 1000 ft vertically and 200 ft horizontally; three of four crews flying without 
TCAS experienced such conflicts. 

The amount of information on the location of other traffic had little effect on the pilots’ performance of 
the maneuvers commanded by the collision-avoidance system. Measured crew responses were similar 
without information regarding traffic location, with limited traffic information, and with continuous 
traffic information. The only measure which showed any difference associated with the level of traffic 
information was peak vertical velocity overshoot. These peaks were not associated with differences in 
root mean square velocity overshoot and did not produce differences in the amount of altitude displace- 
ment during the maneuvers. 

Seven of eight crews provided with a planform display of traffic information used that information at 
least once to maneuver in advance of receiving a maneuver advisory, despite instructions not to do so 
(discussed in detail during their training and which they understood, as indicated by their comments in 
flight). The use of the traffic display to maneuver prior to receiving a resolution advisory must be 
addressed in TCAS training programs. The training must encourage pilots to use all the information 
available to them to maintain a safe distance from other aircraft, but the inaccuracy of bearing and alti- 
tude information on traffic advisory displays must be emphasized. The likelihood that anticipatory 
avoidance maneuvers may compromise the performance of the TCAS in computing the most effective 
avoidance maneuver should also be discussed in training. Pilots must learn to use the system the way 
the designers and its logic intend it to be used, though they also must remember to use their training 
and experience to evaluate situations and take appropriate action to ensure safety of flight. 
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Three turning maneuvers were made to avoid unseen aircraft with transponders but without altitude 
reporting capability (mode A transponders). Crew comments during these maneuvers indicated that 
they were aware that their instructions were not to make such maneuvers, but that they were also 
aware that no resolution or avoidance maneuvers could be generated by TCAS due to the lack of infor- 
mation about the altitude of the intruder aircraft The uncertainty caused in the cockpit by incomplete 
information about such targets suggests the need for mode C (altitude reporting) transposing equip- 
ment in all aircraft likely to interact with TCAS-equipped aircraft, especially m terminal areas where 
aircrew and air traffic controller workload is especially high. 

The lack of apparent learning effects in this experiment may be due to the limited number of 
maneuvers performed by the individual pilots and the crews as a whole. There also may be a con- 
founding effect of fatigue, since duty days were approximately 10 hr, with 8 segments and 6-hr flight 

time. 

The measures of flight experience which differed across display conditions, Boeing 727 flight time and 
recent flight time, did not manifest themselves in pilot performance. Therefore, display condition 
effects were independent of flight crew experience level within the limited range studied. 

These results, obtained in a full-crew, full-mission environment, may be representative of how flight 
crews will behave when the traffic alert and collision-avoidance system is introduced. The findings 
also have fundamental importance with respect to information transfer issues in today’s and future air- 
craft TCAS is a fairly intelligent system; it represents one of the first "smart" systems to be intro- 
duced into air carrier line operations. For that reason, it will be important to observe, in detail, how 
flight crews utilize and interact with it so that the lessons learned can be applied to the design of other 
"smart" systems now under development, and to the interfaces through which such systems will 
interact with flight crews. 

In this limited experiment, the presence of TCAS conveyed marked and significant benefits in terms of 
safety. Significant additional benefits from the presence of planform displays of other traffic were not 
observed in this study, though this must be further evaluated in flight during the TCAS limited installa- 
tion program. 
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16. Abstract 

Pilots* use of and responses to a traffic alert and collision-avoidance system (TCAS II) in simu- 
lated air carrier line operations are described in Volume I. TCAS II monitors the positions of nearby 
aircraft by means of transponder interrogation, and it coomands a climb or descent when conflicting 
aircraft are projected to reach an unsafe closest-point-of-approach within 20-25 seconds. A different 
level of information about the location of other air traffic was presented to each of three groups of 
flight crews during their execution of eight simulated air carrier flights. A fourth group of pilots 
flew the same segments without TCAS II equipment. Traffic conflicts were generated at intervals during 
the flights; many of the conflict aircraft were visible to the flight crews. 

The TCAS equipment successfully ameliorated the seriousness of all conflicts; three of four non- 
TCAS crews had hazardous encounters. Response times to TCAS maneuver commands did not differ as a func- 
tion of the amount of information provided, nor did response accuracy. Differences in flight experience 
did not appear to contribute to the small performance differences observed. Pilots used the displays of 
conflicting traffic to maneuver to avoid unseen traffic before maneuver advisories were issued by the 
TCAS equipment. The results indicate (1) that pilots utilize TCAS effectively within the response times 
allocated by the TCAS logic and (2) that TCAS II is an effective collision avoidance device. 

Volume II contains the appendices referenced in Volume I, providing details of the experiment and 
the results, and the text of two reports written in support of the program. 
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